Bit Error Bound of Maximum-Likelihood Cochannel Interference Cancellation

In order to increase the capacity of TDMA cellular radio networks, the reuse factor may be recduced. However, in this case the system performance is limited by cochannel interference, unless cochannel interference cancellation is done. Therefore, cochannel interference cancellation is a hot topic at the moment. Assuming perfect knowledge of the channel coefficients of the desired user and the interfering users, the set of optimal sequences in terms of the maximum likelihood criterion can be found using joint maximum likelihood sequence estimation (JMLSE) [1]. In this paper, a lower bound on the BEP for the desired user using JMLSE is introduced. It is conceptually based on a genie lower bound originally derived for multiuser detection [2, Chapter 4.3], which is adapted here to JMLSE and which is moreover extended to frequencyselective fading channels and partial-response modulation. Binary modulation is assumed throughout this paper. Using JMLSE, the BEP for the desired user, JMLSE can be lower-bounded by the BEP of the following genie-aided receiver. Assume there exists a genie who knows the sent data sequences of the desired user and all interfering users of length . Given an error matrix that affects the desired user , the genie verifies whether the sent matrix is compatible with or not, i.e., if . If it is, the genie tells the receiver that the correct is in the set . The receiver determines the most likely of the two hypotheses employing the ML criterion, which is equivalent to a simple threshold decision with two decision regions. If , the genie reveals to the receiver, so that no bit error can be made. Because the number of hypotheses is reduced to two for and no errors occur for